Region-attentive multimodal neural machine translation

We propose a multimodal neural machine translation (MNMT) method with semantic image regions called region-attentive (RA-NMT). Existing studies on MNMT have mainly focused employing global visual features or equally sized grid local extracted by convolutional networks (CNNs) to improve performance. However, they neglect the effect of information captured inside features. This study utilizes object detection for and integrates textual using two modality-dependent attention mechanisms. The proposed was implemented verified architectures (NMT): recurrent network (RNN) self-attention (SAN). Experimental results different language pairs Multi30k dataset show that our improves over baselines outperforms most state-of-the-art methods. Further analysis demonstrates can achieve better performance because its feature use.